Fast Transparent Cluster-Wide Paging

In a cluster with a very low-latency interconnect, the remote memory of nodes can serve as a storage that is faster than local disk but slower than local memory. In this paper, we address the problem of transparently utilizing this cluster-wide pool of unused memory as a low-latency paging device. Such a transparent remote memory paging system can enable large-memory applications to benefit from cluster-wide memory resources without compromising application performance while obviating the need for very large local memory or modifications to legacy applications. There was considerable interest in this subject in the 1990’s, which saw a number of remote paging systems built over specialized interconnects such as Myrinet and ATM switches. However, in recent years, this subject has not received the research attention it deserves. With the advent of highly affordable commodity gigabit Ethernet switches in the market, we examine the feasibility and advantages of using gigabit Ethernet to perform fully distributed remote memory paging. We leverage upon our earlier experiences with the Anemone project [11] – a centralized remote paging system – to develop a fully distributed, transparent, Linux-based alternative over gigabit Ethernet to avoid the disk I/O bottleneck. A self-managing pseudo block device module resides in every participating workstation in the cluster and is responsible for a multitude of functions. This module avoids the entire IP stack and user-space overheads, manages the available cluster-wide memory as well as local disk swap devices, eliminates centralized decision making, is completely transparent to the kernel, dynamically adapts to the availability of remote memory, and fully exploits the bandwidth-delay product of the underlying interconnect. Preliminary performance results indicate that the distributed paging system design can yield sequential program execution speedups of up to a factor of 6 when compared to disk-based paging (versus 3 in our in our cenrtalized design) and pagefault latencies in the order of 210 microseconds.